Support dots in target names when writing to an override block - #6148
Merged
ilyakuz-db merged 1 commit intoAug 3, 2026
Merged
Conversation
Contributor
Approval status: pending
|
Collaborator
Integration test reportCommit: 1544e89
8 interesting tests: 4 RECOVERED, 4 SKIP
Top 9 slowest tests (at least 2 minutes):
|
ilyakuz-db
force-pushed
the
configsync/dotted-target-name
branch
from
August 3, 2026 15:43
03fd794 to
04c769a
Compare
ilyakuz-db
force-pushed
the
configsync/split-list-write-back
branch
from
August 3, 2026 16:04
49300b3 to
8b3e0a8
Compare
ilyakuz-db
force-pushed
the
configsync/dotted-target-name
branch
3 times, most recently
from
August 3, 2026 16:32
0df0b5d to
b4c479d
Compare
ilyakuz-db
force-pushed
the
configsync/dotted-target-name
branch
2 times, most recently
from
August 3, 2026 17:26
71e1909 to
8b3fd57
Compare
Co-authored-by: Isaac
ilyakuz-db
force-pushed
the
configsync/dotted-target-name
branch
from
August 3, 2026 19:21
8b3fd57 to
1544e89
Compare
yolocs
pushed a commit
to yolocs/dbcli
that referenced
this pull request
Aug 4, 2026
…hange (databricks#6138) ## Problem A resource's sequence field (e.g. job `tasks`) can be written in two physical YAML regions: the top-level `resources.*` block and a `targets.` override, either possibly in its own included file. Loading merges them into one list and sorts keyed lists by key. Config-sync resolves a change against that **merged** list, then writes it using the merged position, which causes out-of-bound indexing issues or issues with config corruption ## Solution `bundle/configsync/blockindex.go` maps every `dyn.Location` back to the block that owns it (not only a file), so a destination is *looked up*, and `resolveSelectors` rewrites merged positions into block-local ones. Blocks are recovered by re-parsing the contributing files, because selecting a target destroys the distinction (it folds overrides into `resources` and drops `targets`). Re-parsing, rather than reloading via the mutator pipeline, avoids running the bundle's `preinit` script twice per sync. Guiding rule: **correct write > no write > wrong write.** Anything ambiguous is left for a later run. How a destination is picked: - element defined in one block → that block; - element defined in several → the **field** decides, since each field has its own location, so two fields of one task can go to two different blocks; - field itself defined in several → the definition that **won the merge**: the target override, or between two files in the same scope the one loaded later. Writing the shadowed copy would leave the effective value unchanged, so the sync would re-detect the same change on every run; - **removing** anything defined in several blocks → one deletion per definition, since it only disappears once every copy is gone. This applies to a *field* as well as an element: deleting only the winning copy lets the shadowed one take effect, so the next deploy restores the value the user just removed; - **renaming** → a replace of just the key field in every defining block, so other fields stay in their own scope. A key change arrives as an unlinked remove plus add; the two are paired by identity; - brand-new element or field → the block declaring the resource. A field a mutator inserted (e.g. `OverrideCompute` setting `existing_cluster_id`) counts as new: the value is present but carries no location, so no block declares it; - otherwise → left unapplied rather than written to a guess. Also worth knowing: index bookkeeping is per block, so a removal in one block cannot shift positions in another; rename pairing refuses ambiguous matches in both directions; and a target override may legitimately span several files, so a block is identified by (scope, file) ## Tests New acceptance tests, a lot of test cases to cover everything that this refactoring touched, but all are local as we only need to cover config resolution ## Follow-ups Stacked on this branch, each independently reviewable: - **databricks#6146** — report changes that were detected but not written back. Today every drop path is a `log.Debugf` while the command prints the full changeset and exits 0, so a dropped edit is indistinguishable from an applied one. - **databricks#6148** — support a dot in a target name (`targets: {dev.eu: ...}`). Pre-existing on `main`, but it fails the whole command rather than skipping one change. Rebase onto `main` once this merges.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A target name may contain a dot —
targets: {dev.eu: ...}passesbundle validateand deploys. But write-back addresses an override block by buildingtargets.<target>.<field>as a string, which is then parsed back into path segments. A dotted name splits into two keys, so the write targets/targets/dev/eu/...and the command fails outright:Pre-existing, not introduced by #6138:
mainfails the same way with a different message.Solution
Build the prefix as path nodes instead of concatenating text.
structpath.NewPatternStringKeyalready switches to bracket notation for a name that is not a plain field, sodev.eurenders astargets['dev.eu']and round-trips as one key. Both the routed path and the unrouted fallback go through the same helper.Tests
split/dotted_targetedits a field in the override block of a target nameddev.eu. It fails on both engines without this change (non-zero exit, nothing written) and passes with it. No other golden changes.